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FOREWORD 


The Boston Opportunity Agenda has, since Its inception, been focused on building a strong and seamless 
pipeline for students into Boston's highly competitive economy and active civic life. Over the past |0 years, 
it has become clear that in order to achieve our ambitious goals for high school graduation and college 
completion, we need to rethink how we engage students and prepare them for life and work in the 2Ist 
century. As a result, in 2017, the Boston Opportunity Agenda worked with leaders from across the K-12, 
higher education and workforce development fields to create a shared definition of College, Career and 
Life Readiness for students graduating from all schools in Boston—Boston Public Schools (BPS) as well as 


Boston's Catholic and charter schools. 


In order to define readiness, of course, we had to get clear what we are getting our kids ready for. The group 


recognized that the future is increasingly characterized by its dynamism and that any singular vision narrows 


how we expect our graduates will live. So the group focused on the broad forces that are shaping trajectories 
for career, college and life and that influence the capabilities and mindset students will need to succeed in this 


fast-shifting environment. 


With the goal of having every student graduate with the skills, competencies and experience needed 


to select a post-secondary path—be it college, career, social enterprise—and to experience success, 


the definition goes beyond traditional graduation requirements like MCAS achievement and course 

accumulation. Simply put, on a pathway to success graduates must be able to individually SET A VISION for 
themselves, CHART A COURSE to that vision, CHANGE COURSE as necessary, BUILD COMPETENCE 
and WORK WITH OTHERS. The citywide definition was published in October of 2017 and aligned with a 


set of initial metrics recommended by American Institutes for Research (AIR) based on a literature review. 


Since that literature was primarily national in scope, the Boston Opportunity Agenda was interested to see 


whether the metrics would hold true for Boston students, and undertook this study to assess real outcomes 


against the predictive indicators. The BPS, Catholic and charter schools will all use these metrics to track 
student success, but only BPS has accumulated a large enough data set to validate the metrics we selected. 


Therefore the data used for the study is limited to the experiences of a cohort of BPS students. 


Despite that data limitation, this report gives the city of Boston great insight into the strengths and 


£ 


weaknesses of the metrics we have selected as indicators of post-secondary success. These insights will 
inform changes that area high schools make to ensure students have access to the experiences—both in 


and out of school|—that they need in order to flourish. 


The Boston Opportunity Agenda fervently believes that by combining our resources, expertise and influence 


around a single focus, we will have a greater impact on Boston's cradle-to-career educational pipeline. That 


focus includes cultivating in our young people the power to find goals worth seeking, to figure out how to 


reach them and to navigate new situations and develop skills to surmount any barriers they encounter. 


Mike Durkin Laura Perille 
Chair, Boston Opportunity Agenda Interim Superintendent, Boston Public Schools 


@ Boston Opportunity Agenda 


TABLE OF CONTENTS 


5 PURPOSE 

6 SUMMARY OF KEY FINDINGS 

7 BACKGROUND RESEARCH ON PREDICTIVE INDICATORS OF SCHOOL SUCCESS 
8 SAMPLE AND DATA 


9 HIGH SCHOOL INDICATORS OF POST-SECONDARY ENROLLMENT AND ATTAINMENT 
Validating a Core Set of High School Predictive Indicators of Post-Secondary Success 
Examining the Strength of the Indicators 
Distribution of On-Track to Post-Secondary Success Indicators among BPS HS Graduates 
Using the Predictive Indicators to Establish Strategic Points of Intervention 
Students Who Completed Post-Secondary without Any HS Indicators 
Anytime/Anywhere Learning Indicator and Senior Exit Survey Data 


High School Indicators for Two-Year Programs 
20 SUMMARY 
23 REFERENCES 
24 APPENDIX1 


28 APPENDIX 2 


College, Career and Life Readiness @ 


PURPOSE 


he Boston Opportunity Agenda is a partnership among 
the City of Boston, Boston Public Schools, the University of 
Massachusetts Boston, and the city's leading public charities 
along with many other local foundations and philanthropists. 
The partnership's goal is to use tts members’ varied resources 
and expertise in a unified manner to improve the post- 
secondary, career, and life outcomes for all of Boston's children. 
With this larger goal in mind, the Boston Opportunity Agenda 
undertook a project in 2016 to identify and develop a set of 


college and career readiness indicators, for use by stakeholders in 


Boston's K-12 institutions to guide future policy and practice. 


This project is driven by the necessity of having a post-secondary 


degree to achieve career success and life-long opportunities 


in today's knowledge economy. In Boston specifically, half of all 
job vacancies required at least an associate's degree, at the time 
of this project's undertaking. In addition, a typical bachelor’s 
degree holder will earn $1 million more than a high-school 
dropout over the course ofa lifetime. With only 36.5 percent of 
all Boston Public Schools graduates obtaining a post-secondary 
credential within six years of graduating high school, there is a 
stark need in Boston to ensure that more of its youth achieve a 
post-secondary degree and the life-long opportunities that come 
with it. Further, of those high school graduates who do enroll 

in post-secondary schooling, 36 percent require at least one 
remedial class and only 51.3 percent obtain a credential within six 
years of graduating high school. These statistics suggest that even 
when students earn a high school diploma, many are still doing so 
without the skills, knowledge, and attributes necessary to achieve 
post-secondary success. By identifying a valid set of indicators for 
post-secondary success, local practitioners can determine which 
specific criteria their students must meet prior to completing 
high school in order to be prepared for post-secondary success 
and use those indicators to recognize which of their students are 


currently off-track to achieving those goals. 


As the first part of this project, the American Institutes for 
Research (AIR) completed a literature review in 2017, identifying 


those metrics that prior research had determined to be the 


Wn 


trongest indicators of post-secondary success. Based upon the 
results of their review, and following discussion with stakeholders 
at the local level, the Boston Opportunity Agenda along with 
Boston Public Schools were able to put forth four measures that 
are both strong indicators of post-secondary achievement and 


specific to the local context of the Boston school district and Its 


student population. The measures include: 


® maintaining an attendance rate of 94 percent or higher; 
e achieving a cumulative GPA of 2.7 or higher; 


e completion of an internship, job shadowing, or 
community service, as reported by the Boston Private 
Industry Council and self-reported on the Senior Exit 


Survey; and 


° completing the Massachusetts Recommended Core 


Curriculum while also enrolling in at least one AP, IB, 
dual-enrollment or career and technical education 
(CTE) course. 


The purpose of this current analytics project, the results of which 


are reported below, was to empirically validate those identified 
early indicators of post-secondary outcomes, based upon 


longitudinal data for Boston Public Schools students. 
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SUMMARY OF KEY FINDINGS 


ey results emerging from the report's analyses include: 


e The empirical data observed from BPS Class of 2010 


affirm that the core indicators selected by the local 
stakeholders—attendance of 94 percent or higher, 
GPA of 2.7 or higher, and completion of the MassCore 
curriculum and enrollment in an AP course—are an 
ficient and effective set of on-track indicators for 


@} 


college readiness and degree attainment. 


Nearly nine of ten students to earn a post-secondary 
degree had at least one of the indicators, and two- 


thirds had two or more of the indicators. Conversely, 


only |5 percent of students to earn a post-secondary 
degree managed to do so without a realizing a single 


one of the predictive high school indicators. 


The data shows that high school course work, both 


in terms of grade point average and completion of 


the MassCore curriculum, are key to post-secondary 


success. Even where students exhibit lower attendance 
rates, they are still able to enroll and complete post- 
secondary programs at above-average rates if they have 
at least one of the course-related indicators. 


One major caveat is that the strength of these metrics 
as an indicator of post-secondary success Is almost 
entirely related to success in four-year B.A. programs, 
and not related to accomplishments obtained at 


two-year degree granting institutions. For two-year 


programs, a GPA of at least 2.0 and attendance of at 
least 90 percent might serve as the bottom line entry 
point for possible success in obtaining a post-secondary 
degree of any kind. 


While they show strong efficiency in identifying 
students who succeeded in post-secondary schooling, 


the PSAT and SAT scores for this cohort of students are 


less practical than other identified metrics as on-track 


to post-secondary success indicators. Not all students 
took the exams, how the exams are constructed has 


changed over time, and 90 percent of the students who 
did well on the SAT had one or more of the other 
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on-track to post-secondary success indicators, making 
the information provided by the SAT largely duplicative. 


Should the “Anytime/Anywhere Learning” measure 
be used as an indicator, a distinction should be made 
between the different types of extra-curricular 
engagement. Students who reported engaging in 
volunteering or internships enrolled in and completed 
four-year post-secondary programs at higher rates 


than those who did not. However, students who 


reported participation in job shadowing enrolled in and 


completed post-secondary schooling at lower rates 


than the cohort average (though enrolled in two-year 


programs at slightly higher rates). It could be that job 
shadowing as extra-curricular activity is chosen by 
students typically preparing to enter the work force 
rather than continue their education, and thus makes 


£ 


a poor indicator of post-secondary successes. 


The distribution of the identified post-secondary 
success Indicators among BPS graduates in the classes 


of 2010 and 2012 indicates several areas where effective 
interventions could substantially increase the percent 
of graduates with strong odds of post-secondary 
success. First, 40 percent of the high school graduates 

in the Class of 2010 had none of the post-secondary 
success Indicators. Moving future graduates who are 

on course to graduate without any on-track to post- 
secondary success indicators to graduating with one 


or two indicators should have a substantial impact on 
improving post-secondary success rates. Second, there 
is ample room to increase the number of graduates 


who complete the MassCore curriculum and take one 
or more AP classes. This combination produces the 
greatest odds of completing a four-year degree but, at 
east among the cohorts studied, was not achieved by 
the majority of graduates. Finally, though not as clear 
cut, the evidence suggests that success in two-year 
institutions would likely increase as more graduates 
obtain at least a 2.0 GPA and attend high school at 
east 90 percent of the time. 


BACKGROUND RESEARCH ON PREDICTIVE INDICATORS 
OF SCHOOL SUCCESS 


redictive indicator work is theory-driven. It seeks to |. Indicators are empirically created. 
find which school behaviors and experiences at one level of Powerful indicators are identified based on analysis 
schooling, .e., middle grade or high school, are predictive of of longitudinal data that tracks individual student 
later success in school, such as high school graduation or post- progress over time. In essence, indicators use the 
secondary enrollment and completion. While school success experience of previous cohorts to establish that 
rates vary considerably by demographic groups, research has current students have high odds of being on track to 
shown that demographics do not efficiently predict school a desired outcome, and to intervene when students 
success (Gleason and Dynarski, 2002). A more promising show behaviors or patterns of experiences associated 
focus for developing indicators of school success relies on the with non-desired outcomes in prior cohorts. 


theoretical construct of student engagement and experiences 


2. Indicators are simple and easily collected. 
n school (e.g., Fredricks, Blumenfeld, & Paris, 2004). 


Predictive indicators with the greatest utility use 


Predictive indicators of student success are powerful tools at readily available data that are typically maintained by 
the K-12 level because they can potentially alert educators schools—variables such as grades, attendance, and 
to students or groups of students who need some level of course enrollments. They do not necessarily require 
intervention or more robust school experience to stay on complex statistical modeling techniques or access to 
track to high school graduation or post-secondary successes, data from surveys or interviews. 


dentifying the relevant indicators Is just a first step, and the ste 
708 ; ; P P 3. The set of indicators has been refined to 
that is arguably the easiest. Indicators do no good if they are not . . os 
; include a few key variables. |n indicator systems, 
ollowed up by action and further assessment to see whether i 
é a few key indicators are easier for schools to monitor 
the actions taken have helped to keep students on track to long- 
than a large set of predictors. K-12 analyses have 
term goals. However, appropriate action depends in part ona — 
demonstrated that although the underlying issues 


robust set of predictive indicators that provide sufficient time 
that produced good or poor grades or strong or 


to intervene. A strong indicator set has several characteristics: 
weak attendance may be complex and may vary from 


student to student, there are a small number of flags 
that can alert educators to a student being on track 
or potentially falling off-track. By extension, a good 
indicator system also identifies variables that are not 
the strongest predictors of positive and/or less than 


desired school success. 


4. Indicators capture the majority of students 
who achieve desired outcomes as well as 
those who eventually fail to achieve them. 

A good indicator system avoids the ‘1% problem,” or 
indicators that are highly predictive but only identify 


a small percentage of on- or off-track students. 
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SAMPLE AND DATA 


he following analyses are based on a sample of 2,691 
high school graduates from the Boston Public Schools. These 
students were all part of the cohort that began 8th grade in the 
2005-06 school year and whose expected time of graduation 
would have been 2009-10. In 2005—06, 4,587 students began 
with the cohort as 8th graders. By January 2015, 2,691 of these 
students had earned a high school diploma. Thus, our sample 
does not include students who transferred into BPS after 8th 
grade and excludes students who transferred out between 
8th grade and |2th grade. This was done to be able to conduct 
apples-to-apples comparisons of the high school experience for 
students who were enrolled in BPS for their entire high school 
education. Constructing the sample in this way is what enables 
us to calculate comparable GPAs, attendance rates, and course- 


taking patterns, among other key variables. 


It is this group of high school graduates on whom our analyses 
focus. Knowing that the earning of a high school diploma is not 
enough to guarantee future success, we compare the post- 
secondary outcomes of these high school graduates, based on 
their achievement levels at the time of high school completion, 
to determine which criteria in terms of participation, knowledge, 
and curriculum are most necessary to ensure future post- 


secondary SUCCESS. 


Our outcomes focus on both post-secondary enrollment 
and degree completion and are further broken out between 


outcomes for two- and four-year degree granting institutions. 


Data on post-secondary outcomes came from the National 
Student Clearinghouse and followed students through to the 
end of 2017, more than seven years past their expected time 

of graduation. This enables us to examine the outcomes of 
students who both took more than four years to graduate from 


high school and those who did not enroll in post-secondary 


institutions right after high school graduation. Data on students’ 


high school experiences were obtained from the Boston Public 


Schools’ student information systems. Possible indicators taken 


from students’ high school achievements include: their 
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attendance; grade-point average (GPA); achievement levels 

on the Massachusetts Comprehensive Assessment System 
(MCAS); completion of the Massachusetts Recommended Core 
Curriculum (MassCore); enrollment in AP courses and the taking 
of AP exams; and scores from the PSAT and SAT standardized 
examinations for post-secondary enrollment. While our analyses 
examined all of these indicators, including at varying levels of 
achievement, the results below focus on those specific metrics 
identified through the literature review conducted as the first 
part of this project. (Results for the additional measures can 

be found in Appendix |). BPS also provided data on students’ 
background and demographic characteristics, such as gender, 
ethnicity, free/reduced lunch program status, English-language- 


learner status, and special education status. 


Supplemental data was also received from BPS for an additional 
3,344 |2th grade students who graduated from high school in 
the spring of 2012. This additional data enabled us to look at 
data from the BPS Senior Exit Survey administered to graduates 
in the Class of 2012 but not those of 2010, our earlier cohort. 


In particular, we were able to examine students’ self-reported 


engagement in extra-curricular learning activities, such as 


internships, job shadowing, and volunteer work, and compare 


post-secondary outcomes for students who participated in 


such activities. 


HIGH SCHOOL INDICATORS OF POST-SECONDARY 
ENROLLMENT AND ATTAINMENT 


n testing sets of predictive indicators, our first step is to create 
descriptive tables with those metrics showing the percentage 
of students with each characteristic who eventually realized 
the given outcomes (enrolled in post-secondary or attained a 


post-secondary degree). We also calculate the percentage of all 


students with a given outcome who had that characteristic. This 
allows us to see which indicators are, first, associated with a high 
probability of realizing the chosen outcome, and second, are 
shared by a substantial share of students who did achieve the 


outcome. 


By 2018, 77 percent of the high school graduates in the Class 

of 2010 had enrolled in a post-secondary institution, with 

57 percent having enrolled in a four-year institution and 40 
percent having enrolled in a two-year institution (some of 

the students who first enrolled in two-year institutions also 
later enrolled in four-year institutions). These numbers are 
close to national averages for the graduating Class of 2010, as 
reported by the National Student Clearinghouse, for which 

74 percent of students enrolled in a post-secondary program, 
with 44 percent enrolling in a four-year program and 30 percent 
enrolling in a two-year program. The national numbers are 
based on enrollment within the first two years after high school 
graduation, as compared to the seven years for our study 
sample, and thus would likely be a bit higher than they are with 


an equal time period of observation. In terms of post-secondary 


completion, 39 percent of the students in our sample of BPS 
data for the Class of 2010 managed to earn a post-secondary 
degree, with 35 percent earning a degree from a four-year 
institution but only 6 percent earning a degree from a two-year 
program. These numbers are also comparable to national 
averages, for which 41 percent of high school graduates in the 
Class of 2010 completed a post-secondary degree, 33 percent 
from a four-year degree granting institution and only 8 percent 


from a two-year institution. For post-secondary completion, the 


national rates are based upon six years after the expected time 


of high school graduation. 


|. https://nscresearchcenterorg/ourreports/ 


Validating a Core Set of High School 
Predictive Indicators of Post-Secondary 
Success 


Table | categorizes students by the on-track to post-secondary 
success indicators selected by the Boston Opportunity Agenda 
and Boston Public Schools, showing the percentage of each group 
who had enrolled in a post-secondary institution, as well as the 
percentage to earn a degree. For each of the selected indicators, 
students who accomplished them also enrolled in and completed 
post-secondary schooling at higher rates than the cohort average. 
In general, 90 percent of students with these indicators enrolled 
in post-secondary schooling, and roughly 50—60 percent earned 
a degree. Conversely, the indicators were also shared by the 
majority of those students who did attain a post-secondary 


degree. This suggests that, not only are students with those 


indicators more likely to succeed in post-secondary schooling, 


but that in most cases students did not succeed without them, 


and that they are therefore closely tied to possessing the skills 
and knowledge necessary for post-secondary success. The 
exceptions to this are completion of the MassCore curriculum 
and achieving a score of three or higher on an Advanced 
Placement exam. Most students in the cohort did not accomplish 
these academic bars, and thus they are not characteristics shared 
by most of those graduates who went on to succeed in post- 
secondary schooling. However, 79 percent of students who 

did complete the MassCore and take an AP course completed 
post-secondary and this was the highest overall completion rate 
for any indicator. This suggests that this is an area where there is 
both room for improvement and where there are strong odds 
that improvements would lead to higher rates of post-secondary 
completion. Moreover, since simply taking an AP course or an 


AP exam is nearly as strong at predicting post-secondary success 


as is achieving a “qualifying’’ score on an AP exam, and given that 
far more students in the cohort had taken AP courses or an AP 
exam, the specific score of a student's AP exam is not as effective 


an indicator as AP course taking. Thus, tt seems that a good goal 
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in seeking to increase post-secondary attainment would be 


to increase the number of students taking the MassCore and 


enrolling in AP courses. 


The literature review on early indicators of college and career 
readiness previously completed by AIR also highlighted PSAT 


and SAT exam taking as potential indicators. These measures are 


examined in the supplemental analyses presented in Appendix I. 


While they show strong efficiency in identifying students who 


Table 1: Post-Secondary Enrollment and Completion Rates by Predictive Indicator 


ire lanlatiate 


Numbers 
of Students 
with 


Characteristic 


ALL POST-SECONDARY INSTITUTIONS 


% Who 
Enrolled 
in PS 


succeeded in post-secondary schooling, the PSAT and SAT data 
for this cohort of students are based on prior versions of the 
exams, which have since been modified. Future analysis with 
more current SAT and PSAT data should re-evaluate them. 
However, the empirical data taken from BPS students reaffirm 
that the core indicators selected by the local stakeholders— 
attendance of 94 percent or higher, GPA of 2.7 or higher, and 
completion of the MassCore and enrollment in an AP course— 


are an efficient and effective set of indicators. 


% of Total 
Students to 
Enroll in PS 


% Who 
Completed 


PS 


OM) ia Key | 
Students to 
Complete PS 


Attendance >= 94% 1,238 87% 53% 58% 68% 
GPA >= 2.7 1,163 88% 50% 65% n% 
MassCore 716 90% 32% 66% 5% 
AP Course 1,208 90% 52% 62% 11% 
AP Exam 1,148 1% 51% 65% 10% 
AP Exam Score >=3 512 94% 23% 18% 38% 
MassCore + AP 463 92% 21% 19% 35% 
Entire Cohort 2,691 1% 100% 39% 100% 


TWO-YEAR DEGREE GRANT 


ING INSTITUTIONS 


Attendance >= 94% 1,238 3% 38% 1% 48% 
GPA >= 2.7 1,163 26% 29% 5% 31% 
MassCore 716 26% 18% 4% 18% 
AP Course 1,208 30% 3% 6% 40% 
AP Exam 1,148 28% 31% 6% 39% 
AP Exam Score >=3 512 14% 1% d% 6% 

MassCore + AP 463 15% 1% 3% 8% 

Entire Cohort 2,691 40% 100% 6% 100% 


FOUR-YEAR DEGREE GRANTING INSTITUTIONS 


Attendance >= 94% 1,238 15% 61% 53% 11% 
GPA >= 2.7 1,163 81% 63% 61% 1% 
MassCore 716 82% 39% 63% 49% 
AP Course 1,208 81% 64% 58% 5% 
AP Exam 1,148 84% 3% 60% 4% 
AP Exam Score >=3 512 92% 31% 11% B% 
MassCore + AP 463 90% 28% 1% 39% 
Entire Cohort 2,691 51% 100% 35% 100% 
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One major caveat is that the strength of these metrics as an 
indicator of post-secondary success is almost entirely related to 
success in four-year degree granting institutions, and not related 
to accomplishments obtained at two-year degree granting 
institutions. When breaking out the results by institution type, 
we see that for students with the on-track indicators, enrollment 
and attainment rates in two-year programs were actually lower 
than the average of all students from their graduating class put 
together. In addition, the majority of students who enrolled 

in and completed two-year programs did so without having 
reached these levels of academic achievement in high school. 


Examining the Strength 
of the Indicators 


Next, the selected academic indicators, along with student 
demographics, are used in logistic regression models to 
determine the power of each measure in determining a student's 
odds of realizing the given outcome. The demographic measures 
are included as control measures, to see how predictive the 
academic measures are regardless of student demographic 
characteristics. Table 2, below, presents the results of three 
separate series of logistic regression models. In the first set, 


the selected indicators were used each on their own, without 


demographic controls, to predict students’ post-secondary 
outcomes and determine the raw power of each indicator. In the 
second set of models, each indicator was modeled separately, but 
along with the demographic controls, to see their power when 
controlling for other background factors. The third and last set of 
models included all three indicators at the same time, along with 
the demographic controls to see their combined power as a set. 


In the table, for each effect, the odds-ratio presented can be 
interpreted as the odds of enrolling/attaining for a student with 
that characteristic, as compared to odds of |.0 for students 
without that characteristic. Odds above |.0 represent greater 
likelihood of the event occurring, and odds below |.0 mean 

itis less likely. Taking the first result from the table, students 
whose high school attendance rates were 94 percent or higher 
roughly were three times more likely to enroll in post-secondary 
schooling than were students with attendance rates under 94 
percent (3.2 vs. |.0), without controlling for any other factors. 
Or put another way, students with attendance rates of 94 
percent or higher were 220 percent more likely to enroll in post- 
secondary schooling (3.2-1.0 * 100). 


Without controlling for students’ backgrounds, students with 
high attendance were roughly three times more likely to enroll 


in post-secondary and four times more likely to earn a degree. 


Table 2: Logistic Regression Results 


Enrolled Completed Enrolled Completed Enrolled Completed 
PS PS 2-Year 2-Year 4-Year 4-year 
INDEPENDENT EFFECTS WITH NO CONTROLS 
Attendance >= 94% 3" 430%" 0.6*** Il 4.\e** Agee 
GPA >= 27 34 14"** 0.3*** 0.7* 1** 9.0°"* 
MassCore + AP 4.0°** 8.1*** 0.2*** 0.4*"** 9.4e** lala 


INDEPENDENT EFFECTS CONTROLLING FOR STUDENT DEMOGRAPHICS 


Attendance >= 94% 2.8*** 3.6°** one" 1.3 3.55%" 3 
GPA >= 2.7 Lre* 5.ae 0.4*** 0.8 5." 6.1°** 
MassCore + AP 2.6°** 4.3°** 0.3°** 0.5* 5.0°** 4ae"" 
COMBINED EFFECTS CONTROLLING FOR STUDENT DEMOGRAPHICS 

Attendance >= 94% on" 2.1°** I. 1.6" 1.9*** 2.1/°** 
GPA >= 27 1.8*** 3.6°** 0.4*** 0.7 3.8°** 4g 
MassCore + AP 1.7" 24a" 0.4*** 0.5 2.6°** 2.6°** 


Students’ demographics include gender, ethnicity, English language learner status, special education status, 
and free/reduced lunch program status; * significant at p<.05 level; ** p<.01; *** p<.001 
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Students with high GPAs were also three time more likely to 
enroll in post-secondary than were students with low GPAs, 


and they were seven times more likely to complete a degree. 


Students who completed the MassCore curriculum as well as an 
AP course were four times more likely to enroll, and eight times 
more likely to finish. When controlling for students’ demographic 


characteristics, the effects of the three indicators were more 


equivalent, with each making a student roughly three times 
more likely to enroll in a post-secondary program and four to 
five times more likely to attain a degree. While the addition of 
the demographic characteristics lowered the strength of the 
relationships somewhat, each of the three indicators was still 

a strong and statistically significant predictor of future post- 
secondary outcomes, even after having controlled for students’ 


particular backgrounds. 


The combination of the three indicators together in the last 
series of models served to weaken them further individually, but 
this is to be expected given that the three metrics (attendance, 
GPA, course completion) are highly correlated to each other 
and therefore overlap in their relationships to student outcomes. 
Overall, the model including both demographics and the 
selected indicators explains only |O—|5 percent of the variation 
in post-secondary enrollment between students, but correctly 
predicts the outcome for 78.2 percent of students in our sample. 
For post-secondary attainment, the full model explains only 
28—38 percent of the variation between students, and correctly 


predicts the outcome for 76.4 percent of students. 


When breaking out the model results by institution type, we 


again see a drastic difference in the relationship between the 
indicators to post-secondary outcomes for four-year degree 
granting institutions versus those for two-year institutions. 
While the indicators were positive and statistically significant 
predictors of outcomes for four-year programs, they have 
either no relationship to outcomes for two-year programs or, in 
some cases, have significantly negative relationships. Thus, while 
attendance of 94 percent or higher, GPAs of 2.7 or higher, and 
completion of the MassCore and an AP course are strongly 
related to enrolling in and completing a four-year post-secondary 
program, they were not fundamentals for students to pursue or 


complete two-year degrees. 
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Distribution of On-Track to Post- 
Secondary Success Indicators among 
BPS HS Graduates 


That there is utility in using these academic indicators to measure 
whether students are on track to post-secondary goals can be 
seen in Table 3. For each of the three selected indicators, the table 
shows the share of all students in the cohort with that indicator, 
along with the share of post-secondary enrollees and graduates 
with those indicators. The table also presents the percentages 

of all students, post-secondary enrollees, and post-secondary 


graduates with different numbers of the combined indicators. 


Ww 
attendance rates of 94 percent or higher and GPAs of 2.7 or 


=e 


ile roughly half the students in the entire cohort had 


higher, about two-thirds to three-quarters of students who 


completed a post-secondary degree had these indicators. 
Completion of the MassCore curriculum and at least one AP 
course was about twice as common among students with a post- 
secondary degree as It was among the cohort as a whole. Taken 
as a set, two-thirds of students who enrolled in post-secondary 
schooling had achieved at least one of the three indicators. 
Nearly nine of ten students to earn a post-secondary degree 
had at least one or more of the indicators, and two-thirds had 
two or more of the indicators. Conversely, only 5 percent of 
students to earn a post-secondary degree managed to do so 


without realizing a single one of the high school indicators. 


Table 4 shows the post-secondary enrollment and attainment 
rates for students with different numbers of indicators. More 
than half of students with at least one of the identified indicators 
succeeded in earning a post-secondary degree. For students 
with two or more of the indicators, 91 percent enrolled in 
post-secondary schooling and more than two-thirds attained a 
degree. However, for students with none of the indicators, only 


15 percent managed to complete post-secondary schooling. 


Using the Predictive Indicators 
to Establish Strategic Points of 
Intervention 


From the practical perspective of using the above indicators to 
establish an early warning system—with the ability to identify 
those students least on-track to complete post-secondary 
schooling of some kind, absent effective intervention—targeting 


students who exhibit none of the selected indicators seems 


Table 3: Distribution of Students with On-Track to Post-Secondary Indicators 


% of All 


Stidents % of PS % of PS % of 2-Year | % of 2-Year | % of 4-Year | % of 4-Year 
(N=2,691) Sener Completed Eel. Completed nee Completed 
with... (N=2,071) | (N=1,059) | (N=1,066) | (N=I71) (N=1,516) | (N=929) 
DATA ON EACH INDICATOR 
Attendance >= 94% 41% 53% 68% 39% 48% 61% 1% 
GPA >= 27 44% 50% n% 29% 31% 63% 1% 
MassCore + AP 18% 44% 35% 1% 8% 28% 39% 
NUMBER OF INDICATORS 
0 Indicators 40% 33% 15% 50% 4% 22% 12% 
| Indicators 25% 26% 22% 29% 21% 26% 21% 
2 Indicators 23% 26% 31% 17% 30% 32% 38% 
3 Indicators 12% 15% 26% 3% 2% 20% 29% 
| or more 60% 61% 85% 50% 58% 18% 88% 
2 or more 35% 4% 63% 20% 32% 52% 61% 


Table 4: Outcomes, by Number of Indicators 


% PS Enroll % PS % 2-Year % 2-Year % 4-Year WM (-¥:1 9 

: Complete Enroll Complete Enroll Complete 
0 Indicators 63% 15% 49% 1% 31% 10% 
| Indicator 80% 34% 46% 1% 58% 29% 
2 Indicators 89% 64% 31% 8% 81% 58% 
3 Indicators 93% 84% 10% 1% 93% 84% 
| or more 86% 56% 33% 6% 14% 51% 
2 or more 1% 11% B% 6% 85% 61% 


feasible and practical. Of the 2,691 students in the cohort 
of graduates, 1,083 displayed none of the indicators and just 


points mean that any programs targeting such students would 
have caught the correct students in at least four of five cases, 
15 percent of these students successfully completed a post- while at the same time addressing the majority of students in 
secondary degree. Considering that nearly two-thirds of them the cohort who failed to achieve post-secondary success, thus 
did enroll in a post-secondary program, it shows that there wasting relatively little of any implemented resources/personnel 
remains a lot of room for improvement as they are enrolling in and making a substantial impact on the overall problem. The 
post-secondary schooling but just not succeeding. In addition, analysis also reveals a somewhat larger set of students needing 
this group represents just over a third of the entire cohort (40%), | moderate interventions—those with a single indicator. 
while capturing over half (56%) of all the students who failed to 


earn a post-secondary degree. Taken in combination, these two 


College, Career and Life Readiness ® 


Table 5: Outcomes, by Combinations of Indicators 


% PS yw a (-¥-1 9 yw ey (-¥-1 g Vie (-¥- 1g % 4-Year 


Complete Enroll Complete Enroll Complete 


High Attendance 
High GPA 
MassCore/AP 
(N=324) 


High Attendance 
High GPA 
No MassCore/AP 
(N=472) 


High Attendance 
Low GPA 
MassCore/AP 
(N=44) 


High Attendance 
Low GPA 
No MassCore/AP 
(N=332) 


93% 84% 10% I% 93% 84% 


88% 63% 31% 8% 19% 56% 


1% 66% 32% 16% 86% 59% 


19% 21% 56% 8% 49% 21% 


Low Attendance 
High GPA 
MassCore/AP 
(N=69) 


90% 1% 22% 3% 87% n% 


Low Attendance 
High GPA 
No MassCore/AP 
(N=265) 


81% 43% 36% 5% 69% 38% 


Low Attendance 
Low GPA 
MassCore/AP 
(N=25) 


88% 40% 44% 4% n% 36% 


Low Attendance 
Low GPA 
No MassCore/AP 
(N=996) 


65% 16% 51% Th 32% 1% 


Cohort Average 


(N=2,691) 1% 39% 40% 6% 51% 35% 


Table 5, above, displays post-secondary enrollment and lower attendance rates, they are still able to enroll in and 
attainment outcomes for students with varying combinations complete post-secondary programs at above average rates 

of the high school academic indicators. In a sense, the table if they have at least one of the course-related indicators. It 
presents the trade-offs in values of realizing one indicator versus is only in those cases where students are missing both the 
another. The table shows that course work, both in terms of course-related indicators that they experience post-secondary 
grade point average and completion of the MassCore curriculum, | outcomes at rates lower than the cohort average. 


is key to post-secondary success. Even where students exhibit 
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Students Who Completed Post- 
Secondary without Any HS Indicators 


Table 6, below, looks more closely at those students who 
managed to earn a post-secondary degree despite exhibiting 
none of the three predictive indicators in high school. The table 
compares descriptive statistics for students with none of the 
indicators who earned a degree, and those with no indicators 
who failed to earn a degree, in order to contrast the two 
groups and highlight any possible differences. The table further 
compares to two additional groups—those students with all 


three of the high school indicators, and the cohort as a whole. 


While the data available for analysis are limited, there do seem 
to be some general differences between those students with 


none of the indicators who earn a post-secondary degree and 
those who do not, as well as those with predictive indicators 
who obtained degrees. Of the 164 high school graduates from 
the Class of 2010 who earned a post-secondary degree without 
having a high school predictive indicator of post-secondary 


success, /| (or nearly half) earned a two-year associate's degree. 


Thus, the first thing that stands out about this group Is that 

in contrast to students with one or more of the predictive 
indicators they had a much higher rate of obtaining two-year 
degrees. Even so, the majority of students who earned a post- 
secondary degree without having high school attendance of 
94 percent or higher, a GPA of 2.7 or greater, or taking the 
MassCore and one AP class obtained a four-year B.A. degree. 


When compared with students who did not earn post-secondary 


Table 6: Descriptive Statistics for HS Graduates with None of the Post-Secondary Indicators 


No Indicators ‘ Students 
Buk No Indicators with all 
and No 3 Total Cohort 
Earneda PS Deoree PS Indicators (N=2,691) 
rsdegee | Tyson | (tesza)y” |? 
% Female 60% 48% 59% 3% 
% FRL 68% 16% 4% 66% 
% Spec. Ed. 8% B% 1% 14% 
% ELL 6% 1% 0% 6% 
% Asian 5% 3% 39% 12% 
% Black Al% 55% 15% B% 
% Hispanic 40% 33% 8% 29% 
% Other 0% <I% 2% 1% 
% White 13% 9% 31% 16% 
Average GPA 2.0 1.6 3.8 2.6 
Average Attendance Rate 88% 85% 91% 1% 
% Proficient 10th Grade MCAS ELA 10% 44% 91% 66% 
% Proficient 10th Grade MCAS Math 65% 4% 98% 68% 
% with AP Course 31% 14% 100% 4% 
% with AP Exam 21% 10% 11% 22% 
% taking PSAT 88% 81% 98% 89% 
% taking SAT 68% 46% 98% 1% 
% with MassCore 8% 1% 100% 28% 
% with MassCore + AP 0% 0% 100% 18% 
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degrees though, students who succeed in post-secondary 
without any predictive indicators had modestly higher GPAs 

and rates of attendance and were much more likely to score 
proficient on |Oth grade MCAS tests. Therefore, while those 
who did not earn a degree struggle in almost every academic 
aspect, those students who beat the odds and earned a degree 
more closely resemble the cohort average both academically and 
demographically. A better understanding of how such students 
manage to complete post-secondary schooling despite only 
modest success in high school likely requires a more detailed and 
qualitative analysis of such students, taking a closer look at their 


post-secondary success indicator by looking at post-secondary 
outcomes for students who completed an internship, job 
shadowing, or community service, as self-reported on the BPS 
Senior Exit Survey. These are the variables local stakeholders 
concurred might be able to capture the value of Anytime/ 
Anywhere Learning for post-secondary success. Table 7 shows 
the post-secondary enrollment and completion rates for the 
Class of 2012 broken out by their responses on the survey to a 
question regarding extra-curricular learning. Overall, the post- 
secondary enrollment and completion rates for the Class of 2012 
are almost identical to those for the Class of 2010 (given a few 


backgrounds and socio-emotional characteristics. ; ; 
percentage points lower, corresponding to the two fewer years 


the cohort has had to achieve the outcomes). 


Anytime/Anywhere Learning Indicator 
and Senior Exit Survey Data 


Students who were engaged in Anytime/Anywhere Learning 
did enroll in and complete post-secondary programs at higher 
Using the supplemental data for 2012 high school graduates, we rates than the cohort average. Of students who reported being 


were able to look at the possibility of establishing an additional engaged in job shadowing, an internship, or volunteering, 


Table 7: Outcomes, by Anytime/Anywhere Learning 


Numbers 
of Students 
with 
Characteristic 


% of Total 
Students to 
Complete PS 


% of Total % Who 
Students to Completed 
Enroll in PS PS 


% Who 


ir lelaxlatjate Enrolled 


in PS 


ALL POST-SECONDARY INSTITUTIONS 


Job Shadowing 322 N% 9% 4% Th 
Internship 595 86% 20% Al% 21% 
Volunteer 1,538 84% 51% 46% 62% 
Any of the Three 2,455 83% 19% 4% 90% 
Class of 2012 3,344 71% 100% 34% 100% 
Job Shadowing 322 48% 12% 6% 13% 
Internship 595 31% 18% 4% 14% 
Volunteer 1,538 33% Al 5% 49% 
Any of the Three 2,455 36% 11% 5% 16% 
Class of 2012 3,344 38% 100% 4% 100% 
Job Shadowing 322 45% 8% 19% 6% 
Internship 595 66% 21% 38% 22% 
Volunteer 1,538 68% 51% 4% 64% 
Any of the Three 2,455 65% 86% 38% 2% 
Class of 2012 3,344 55% 100% 30% 100% 
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83 percent enrolled in post-secondary schooling and 42 percent 
of the total completed a degree. However, the distinction is not 


as strong as for students with the other high school indicators. 


n comparison, students with attendance of 94 percent or 
higher, GPAs of 2.7 or higher, and students who completed the 
MassCore curriculum as well as an AP course typically enrolled 


at rates of 90 percent and two-thirds managed to earn a degree. 


(Ca) 


hould the Anytime/Anywhere Learning measure be used 


as an indicator by BPS, a distinction might be made between 
the different types of extra-curricular engagement. When 
breaking out student responses for the three different types of 


activities, we see that students who reported participating in job 


shadowing actually enrolled in and completed post-secondary 
schooling at lower rates than the cohort average (but enrolled in 
two-year programs at slightly higher rates). It could be that job 
shadowing as an extra-curricular activity is engaged in typically 
by students preparing to enter the work force as opposed 

to planning to continue their education, and thus is a poor 


indicator of post-secondary successes. Students who reported 


involvement in either internships or volunteering experienced 
similar post-secondary enrollment and completion rates, though 
many more students in the cohort were engaged in volunteering 
than internships. A future Anytime/Anywhere Learning indicator 
may best rely only on engagement in these two activities. 
Moreover, there is also an unresolved issue. It cannot be 
determined from the data available whether the experience of 


volunteering and interning develops skills not captured by the 


other predictive indicators but valuable in succeeding in post- 


secondary institutions, or whether the students who participate 


in these activities are those enrolling in competitive colleges, 
which both seek out such participation and tend to have higher 


completion rates. 


In examining the other items on the Senior Exit Survey, the only 
significant differences between students who attended post- 
secondary schooling and those who did not, in terms of their 
high school experiences, was their participation and involvement 
in extra-curricular activities. Otherwise, students reported 
remarkably similar experiences as to how they perceived 

the quality of their high school instruction, regardless of their 


post-secondary outcomes. Similarly, in terms of school climate, 


graduates reported highly similar opinions of their schools and its 


value in their lives, regardless of whether they attended no post- 
secondary schooling, enrolled in a two-year program, or enrolled 


in a four-year program. That students had such similar responses 


on almost all survey items, no matter their post-high school 
destination, also suggests that the Senior Exit Survey items 


would not be very predictive of post-secondary outcomes. 


High School Indicators for Two-Year 
Programs 


Since the on-track to post-secondary success indicators that 
worked very well for four-year degrees did not work for 
two-year degrees, we attempted to see whether a separate 

set of indicators could be found that would be predictive of 
earning two-year degrees. With the data on hand, seen in Table 
8, we were not very successful. Perhaps a GPA of at least 2.0 and 
attendance of at least 90 percent could serve as the bottom line 
entry point for possible success in obtaining a post-secondary 


degree. This is the threshold that was also almost achieved by 


those students who did not have any on-track indicators to a 
four-year degree but did manage to receive a post-secondary 


degree. 


One reason why tt is difficult to establish predictive indicators 

of post-secondary success in two-year degree programs is 

that, overall, the success rate of BPS students from the Class 

of 2010 who enrolled in these programs was very low. This is 

an important finding, itself. Forty percent of the high school 
graduates in the cohort enrolled in a two-year program, but 
only 6 percent of the cohort earned a two-year diploma. This 
means that just 15 percent of the BPS high school graduates who 
enrolled in a two-year degree program attained a diploma from 
a two-year institution. One possible explanation was that such 
students only enrolled in two-year institution in order to build 
up their marks before then transferring to a four-year program. 
However, the data from our sample does not support this 
hypothesis. Of those students who first enrolled in a two-year 
program, later enrollment in a four-year program was higher for 
students who completed the two-year program (63%) than for 
those who failed to earn the two-year degree (46%). Of those 
students who did enroll in a two-year college, only 16 percent of 
them completed a post-secondary degree of any kind (two-year 
or four-year). Given that the national rates of enrollment and 
completion for two-year programs were similar (30% and 8%, 
respectively) for the same cohort, this question is a larger one, 
beyond Boston Public Schools alone, and one that would require 


further and more specific exploration to answer. 
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Table 8: Two-Year College Outcomes, by GPA and Attendance Categories 


Enrolled 
in 2-Year 
Program 


Completed 
2-Year 
Program 


GPA 


Categories 


Enrolled 
in 2-Year 
Program 


Attendance 
Categories 


<80% 


Completed 
yy (1g 
Program 


1.0 — 1.49 48% 5% 80 — 84.9% 3% 1% 
1.5 — 1.99 53% 5% 85 — 89.9% 46% 1% 
2.0 — 2.49 53% 1% 90 — 93.9% 46% 6% 
2.5 — 2.99 40% 8% 94 — 96.9% 34% 6% 

97 — 100% 31% 8% 


4.0 — 5.00 


Cohort Average 


Table 9 might provide some direction for this analysis. It shows 
the descriptive statistics for those 17] BPS high school graduates 
in the Class of 2010 who enrolled in and completed a degree at 
a two-year college. A few outcomes stand out. First, two-year 
degree completers are more female than male by a ratio of 2 to 
|. Second, they are much more likely than four-year degree com- 
pleters to be Black or Hispanic. Three fourths of the two-year 
degree graduates attended high school at least 90 percent of the 
time but a quarter of the two-year graduates were chronically 
absent during high school. Two thirds of them scored proficient 
on 10th grade MCAS, with 40 percent enrolling in an AP course. 
What varied greatly was their GPAs. Roughly a quarter had high 
GPAs of 3.0 or greater, but an almost equal number had low 
GPAs of less than 2.0 (with 50 percent having GPAs between 2.0 
and 2.9). This indicates that most attended school often enough, 
and had at least one measure of some academic strength—be 

it GPA, or 10th grade MCAS, or taking an AP course—even if 
their overall high school record also indicated areas and periods 
of struggle. 
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Cohort Average 


Table 9: Descriptive Statistics for Students 
Who Completed a Degree at a Two-Year College 


Female 61% 
Male 33% 
FRL 14% 
Not FRL 26% 
Spec. Ed. 15% 
Not Spec. Ed. 85% 
ELL 1% 
Not ELL 89% 
Asian 6% 
Black 44% 
Hispanic Al% 
Other 1% 
White 9% 
Att <80'% 5% 
Att >=80% & <85% 8% 
Att >=85% & <90% 15% 
Att >=90% & <95% 32% 
Att >=95'% Al% 
GPA < 2.0 23% 
GPA >=2.0 & <2.5 29% 
GPA >=2.5 & <3.0 21% 
GPA >=3.0 & <3.5 14% 
GPA >=3.5 13% 
Completed MassCore 18% 
Did Not Complete MassCore 82% 
Completed AP course 40% 
Did Not Complete AP course 60% 
Completed MassCore + AP Course 8% 
Did Not Complete MassCore + AP Course 92% 
Proficient on 10th Grade MCAS ELA 62% 
Not Proficient on 10th Grade MCAS ELA 38% 
Proficient on 10th Grade MCAS Math 62% 
Not Proficient on 10th Grade MCAS Math 38% 
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SUMMARY 


o achieve career success in today’s job market, most 
students will require at least some level of post-secondary 
schooling. A high school diploma alone is no longer enough to 
open the door to most career and life-long opportunities. In 
order to support the future success of students coming through 
Boston Public Schools as well as Boston's Catholic and charter 
schools, the Boston Opportunity Agenda, a partnership among 
the key stakeholders in the City of Boston's education system, 
undertook an effort to identify and develop a set of college and 
career readiness indicators for use in Boston's K-12 institutions. 
The expectation is that those indicators can be used to identify 
students who are not on track to achieve post-secondary success, 
in order to intervene with them at a time-point in their high 


school education that is early enough to get them back on track. 


This report with its findings about indicators and outcomes is 


intended to support implementation of the College, Career and 
Life Readiness framework. Together, they can guide all schools 
as they adapt and redesign operations, curriculum, and learning 
opportunities so that higher proportions of Boston students can 


graduate from high school prepared to enroll in and complete a 


meaningful post-secondary credential. 
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Beginning with a literature review of prior research on early 
warning indicators conducted by the American Institutes 

for Research (AIR), and following further discussion with 
stakeholders at the local level, the Boston Opportunity Agenda 
identified four measures deemed to be both strong indicators 
of post-secondary achievement and specific to the local Boston 


context. The measures include: 


® maintaining an attendance rate of 94 percent or higher; 
e achieving a cumulative GPA of 2.7 or higher; 


* completing an internship, job shadowing, or community 
service, as reported by the Boston Private Industry 
Council and self-reported on the Senior Exit Survey; 


and 


* completing the Massachusetts Recommended Core 
Curriculum (MassCore) while also enrolling in at least 
one AP, IB, dual-enrollment, or career and technical 


education (CTE) course. 


The Everyone Graduates Center at Johns Hopkins University 
used two longitudinal samples of data for the high school 


graduates of 2010 and 2012, respectively, to empirically test and 


validate the strength and reliability of those indicators in terms of 


predicting students’ future post-secondary outcomes. 


The data from the BPS Class of 2010 affirmed that three of the 
core indicators selected by the local stakeholders—attendance of 
94 percent or higher, GPA of 2.7 or higher, and completion of 
the MassCore and enrollment in an AP course—are efficient and 
effective indicators for college readiness. Nearly nine of ten 
students to earn a post-secondary degree had at least one or 
more of the indicators, and two-thirds had two or more of the 
indicators. Conversely, only |5 percent of students to earn a 
post-secondary degree managed to do so without a realizing a 
single one of the high school indicators. Analysis showed that of 
all the indicators, the two course work-based metrics (grade 


point average and completion of the MassCore curriculum) best 


foretell post-secondary success. Even where students exhibit 
lower attendance rates, they are still able to enroll and complete 
post-secondary programs at above average rates if they have at 


least one of the course-related indicators. 


The analyses in this report did evaluate other measures as 
potential indicators (Appendix |); however, none proved to be 


stronger and more practically useful predictors than those four 


measures. This is even true for PSAT and SAT scores. Both, in 


themselves, are predictive of post-secondary success. However, 
many students did not have one or the other score, and the 
test Itself is only taken at one point in time, limiting its utility 
in measuring changes in on-track status for students. Finally, 


the higher the SAT or PSAT score, the more predictive it was, 


but also the more likely that students had one or more of the 
other predictive indicators. Ninety percent of students with 
SAT sub-scores of 500 or higher had one or more of the other 
selected high school predictive indicators of post-secondary 


success, making the information provided by the SAT redundant. 


Data from the BPS Class of 2012, specifically graduates’ 
responses on the Senior Exit Survey, allowed us to test the 
fourth selected indicator: students’ participation in an internship, 


job shadowing, or volunteering. Results showed that, overall, 


the Anytime/Anywhere Learning indicator is a modest predictor 
of post-secondary outcomes. The results also showed that 

a distinction should be made between the different types of 
extra-curricular engagement. Students who reported engaging 
in volunteering or internships enrolled in and completed four- 
year post-secondary programs at high rates. However, students 
who reported participation in job shadowing enrolled in and 
completed post-secondary schooling at lower rates than the 
cohort average (but enrolled in two-year programs at slightly 
higher rates). It could be that job shadowing as an activity is 
engaged in typically by students who are planning to enter the 
work force rather than by those preparing to continue their 


education, and thus makes a poor indicator of post-secondary 


success. It also remains unknown whether it is the benefits 
of participating in volunteering and internships that result in 
higher post-secondary outcomes or whether the relationship is 


driven by other characteristics of students who seek out these 


opportunities, or the post-secondary institutions that value them. 


In examining the other items on the Senior Exit Survey, 


graduates reported remarkably similar experiences as to 


how they perceived the quality of their high school instruction 


and school climate, regardless of whether they attended no post- 
secondary schooling, enrolled in a two-year program, or enrolled 
in a four-year program. That students had such similar responses 
on almost all survey ttems, no matter their post-secondary 


outcomes, suggests that the remaining Senior Exit Survey 


items would be of limited value in predicting post-secondary 


outcomes. 


The distinction between results for two-year programs and 


those for four-year programs prompts a caveat applicable across 


all of the report's findings. In general, the strength of these 
metrics as an indicator of post-secondary success is almost 
entirely related to success in four-year programs, and not related 
to accomplishments obtained at two-year degree granting 


institutions. The four selected indicators, as well as other sets 


of indicators, were ineffective at predicting post-secondary 
outcomes for two-year programs. As a starting point, a GPA 

of at least 2.0 or success on a measure of academic skill and 
attendance of at least 90 percent might serve as the bottom line 
entry point for possible success in obtaining a post-secondary 


degree of any kind. However, further research into two-year 


programs—both what drives students toward them and what 


enables success therein—remains an area for future research. Of 


specific interest is that while 40 percent of high school graduates 
in the cohort enrolled in a two-year program, only 6 percent of 
the BPS high school graduates from the Class of 2010 earned a 
two-year diploma. Given that the national rates of enrollment 


and completion for two-year programs were similar (30% and 


8%, respectively) for the same cohort, this question is a larger 
one, beyond Boston Public Schools alone, that would require 


further and more specific exploration to answer. 


Another question that remains unanswered, and that could 
merit further and more specific examination, is how some 
students manage to complete post-secondary schooling despite 
struggling through high school. That some students manage 

to succeed in post-secondary without any of the selected high 
school indicators shows It is possible to beat the odds. An 
understanding of how this occurs, however, requires a more 
detailed and qualitative analysis of such students, including 
taking a closer look at their backgrounds and socio-emotional 


characteristics. 


Finally, the validation of a succinct set of on-track indicators to 


college readiness and attainment, and measurement of their 


frequency among BPS high school graduates, points to some 
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places where interventions are likely to increase the number of 
high school graduates with strong odds of success in four-year 


degree granting post-secondary institutions. They include: 


a. Helping students who are on course to graduate with 
no predictive indicator of post-secondary success to 
gain one or two, as well as helping those with one to 


gain a second. 


b. Increasing the number of students who complete the 


MassCore and take at least one AP class. 


Though not quite as clear-cut, the available evidence also 
indicates that success in two-year institutions could be increased 


if students achieve at least a 90 percent attendance rate, and a 
GPA of 2.0 or above. 


In Boston, half of all job vacancies required at least an associate's 
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degree at the time of this project's undertaking. Yet only 36.5 


percent of all Boston Public Schools graduates are obtaining 


a post-secondary credential within six years of graduating 


from high school. The gap between current o 


known needs, combined with the importance 


secondary degree to achieve career success in 


utcomes and 
of having a post- 


today’s knowledge 


economy, is what drives the Boston Opportunity Agenda. This 


report helps identify where strategic opportu 


nities exist to close 


the post-secondary preparation gap in Boston, and where more 


learning is needed. Both are urgently warranted by the stark 


need in Boston to ensure that more of its youth achieve a post- 


secondary degree and the life-long opportunities and benefits 


that come with It. 
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APPENDIX 1: 


ADDITIONAL PREDICTIVE INDICATORS EXAMINED 


The tables below present results similar to those from Table 

| in the above report, for other indicator measures that were 
tested, based upon all student level data that was available for 
analysis. Some of these measures, such as GPA and attendance, 
are similar to those measures examined more closely in the 
report, but with different cut-off levels that were not as powerful 
in identifying those students who experienced success in post- 
secondary schooling. Other measures are based entirely upon 


different types of student data, primarily test scores. 


While students’ PSAT and SAT scores both appear to be 
efficient at predicting post-secondary outcomes, a practical 
problem is that both tests have undergone substantial scoring 
changes between now and the time in which our cohort of 
graduates took the tests (primarily in 2007 to 2010). At the 
time of our observed data, the PSAT included three sections, 
but has since been reduced to two as writing and reading have 
been combined, and the scales of the test scores have also 


undergone a change. In the table below, we use the National 


Merit Scholarship Corporation (NMSC) selection index scores 
from the PSAT. While the scales for the NMSC scores have also 


undergone a shift, they remain on a somewhat similar range 


and so are the best comparable scores for old PSAT scores to 
current. The scores from the SAT remain on the same scale 
now as when our student sample took the test, but the SAT 
has also been reduced from three to two sections. Thus, for 
both tests, while students’ scores were efficient predictors of 
their post-secondary success (not surprising given their link to 
post-secondary admissions) the changes in the tests’ structures 
and scoring ranges mean that the results below are not directly 


translatable to current cohorts of BPS students. They can, 


however, provide preliminary evidence, and further analysis 
of more recent PSAT and SAT scores could be conducted to 


establish more current cut-off levels and on-track indicators. 


It is worth noting that of those students with scale scores of 400 
or more on any of the SAT subtests (Math, Critical Reading, 
Writing), more than 80 percent had at least one of the other 
BPS selected indicators (GPA >= 2.7, attendance >= 94 percent, 


or completion of the MassCore curriculum plus at least one AP 
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course). Of those with SAT sub-scores of 500 or higher, more 
than 90 percent had at least one of the other selected high 
school indicators. Thus, most students with high SAT scores are 
already captured among the larger group of students with the 


selected indicators. 


A further problem with using SAT scores, as well as most test 
scores, such as the 8th grade MCAS, is that many students did 
not take the tests and thus have missing data for those indicators. 
This further limits their practical utility. In cases where test 
scores prove to be excellent on-track indicators, they might 

be considered secondary or supplementary indicators to the 
primary set confirmed in the above report, which include 


measures for which all BPS high school students have data. 


Numbers % Who % of Total % Who ‘% of Total 


Characteristic of ae: Enrolled Students to | Completed | Students to 
Charackerictic in PS Enroll in PS PS Complete PS 
Attendance >= 90% 1,889 83% 16% 49% 87% 
GPA >= 2.5 1,336 87% 51% 62% 19% 
GPA >= 3.0 917 1% 4l% 1% 62% 
GPA >= 3.5 547 93% 25% 19% Al% 
8th Grade MCAS ELA 1,647 85% 10% 51% 82% 
8th Grade MCAS Math 802 92% 31% 66% 52% 
8th Grade MCAS Science 280 1% B% 68% 18% 
10th Grade MCAS ELA 1,711 86% B% 53% 87% 
10th Grade MCAS Math 1,747 85% 15% 52% 87% 
eli Merit Selection Index 1558 36% 65% 55% 81% 
ea a Merit Selection Index 319 92% 36% 69% 54% 
ere Merit Selection Index 368 92% 16% 16% 2% 
Took the SAT 1,901 85% 18% 49% 88% 
SAT Critical Reading >=300 1,747 87% BY 52% 87% 
SAT Critical Reading >= 400 1,173 90% 51% 62% 69% 
SAT Critical Reading >=500 624 92% 28% B3% 43% 
SAT Mathematics >=300 1,777 86% 4% 52% 87% 
SAT Mathematics >=400 1,372 88% 58% 59% 16% 
SAT Mathematics >=500 789 92% 35% 10% 52% 
SAT Writing >=300 1,757 87% BY 52% 87% 
SAT Writing >=400 1,142 1% 50% 64% 69% 
SAT Writing >=500 598 B% 21% 16% B% 
Entire Cohort 2,691 11% 100% 39% 100% 
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Umer %Who | %ofTotal | %Who | %of Total 


Characteristic of seeds Enrolled Students to | Completed | Students to 
Characteristic in PS Enroll in PS in) Complete PS 

Attendance >= 90% 1,889 37% 67% 1% B% 
GPA >= 2.5 1,336 29% 36% 6% 48% 
GPA >= 3.0 917 BY 20% 5% 21% 
GPA >= 3.5 547 19% 10% 4% 13% 
8th Grade MCAS ELA 1,647 35% 56% 5% 55% 
8th Grade MCAS Math 802 5% 20% 4% 21% 
8th Grade MCAS Science 280 19% 5% 4% 6% 
10th Grade MCAS ELA ITI 34% 58% 6% 62% 
10th Grade MCAS Math 1,747 35% 60% 6% 62% 
ee Merit Selection Index 1,558 3% 48% 6% 51% 
a Merit Selection Index 819 2% 171% 4% 18% 
eo Merit Selection Index 368 14% 5% 1% 3% 
Took the SAT 1,901 38% 68% 1% 74% 
SAT Critical Reading >=300 1,747 36% 59% 1% 68% 
SAT Critical Reading >=400 1,173 28% 31% 5% 35% 
SAT Critical Reading >=500 624 19% 11% 3% 12% 
SAT Mathematics >=300 1,777 36% 61% 1% 68% 
SAT Mathematics >=400 1,372 31% 39% 6% 46% 
SAT Mathematics >=500 789 2% 16% 4% 19% 
SAT Writing >=300 1,757 36% 60% 1% 10% 
SAT Writing >=400 1,142 28% 30% 5% 36% 
SAT Writing >=500 598 38% 10% 1% 10% 
Entire Cohort 2,691 40% 100% 6% 100% 
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pT %Who | %ofTotal | %Who | %of Total 


irlelanlatiate of irra Enrolled Students to | Completed | Students to 
Characteristic Tis) Enroll in PS PS Complete PS 
Attendance >= 90% 1,889 67% 83% 44% 90% 
GPA >= 2.5 1,336 19% 10% 571% 83% 
GPA >= 3.0 917 86% 52% 67% 67% 
GPA >= 3.5 547 89% 33% 16% 45% 
8th Grade MCAS ELA 1,647 1% 80% 41% 86% 
8th Grade MCAS Math 802 85% 46% 63% 56% 
8th Grade MCAS Science 280 86% 16% 65% 20% 
10th Grade MCAS ELA 71 BY” 84% 48% 90% 
10th Grade MCAS Math 1,741 11% 84% 41% 1% 
ee Merit Selection Index 1558 14% 15% 51% 85% 
ala Merit Selection Index 319 87% 46% 67% 59% 
cee Merit Selection Index 368 90% 0% 15% 30% 
Took the SAT 1,901 68% 85% 44% 90% 
SAT Critical Reading >=300 1,747 1% 83% 41% 89% 
SAT Critical Reading >= 400 1,173 82% 63% 59% 4% 
SAT Critical Reading >=500 624 89% 36% 10% 41% 
SAT Mathematics >=300 777 1% 83% 41% 89% 
SAT Mathematics >=400 1,372 11% 69% 54% 80% 
SAT Mathematics >=500 789 87% 45% 67% 57% 
SAT Writing >=300 1,757 1% 83% 41% 89% 
SAT Writing >=400 1,142 82% 62% 60% 74% 
SAT Writing >=500 598 90% 35% BY” 41% 
Entire Cohort 2,691 51% 100% 35% 100% 
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APPENDIX 2: 


DESCRIPTIVE STATISTICS FOR STUDENTS WHO DID NOT 
ENROLL IN POST-SECONDARY SCHOOLING IMMEDIATELY 
AFTER GRADUATING FROM HIGH SCHOOL 


The table below compared basic statistics for high school grad- As seen below, students who failed to immediately enroll in 
uates who enrolled in post-secondary schooling immediately post-secondary schooling face many more challenges in terms 
after high school graduation with those who enrolled in post-sec- of demographic backgrounds (poverty, minority status, ELL, 
ondary schooling at a later date. Here immediate enrollment in special education) and also struggled significantly more in terms 
post-secondary schooling is defined as enrollment by Decem- of all academic performance measures (attendance, GPA, course 
ber 3lst of 2010, the year of expected high school graduation. completion, test proficiency). 


Students who graduated Students who graduated 
HS but did not immediately HS and immediately 
enroll in PS enrolled in PS 
(N = 400) (N=1505) 
% Female 60% 56% 
% FRL 15% 60% 
% Spec. Ed. 11% 8% 
% ELL 9% 4% 
% Asian 5% 11% 
% Black 41% 38% 
% Hispanic 39% 4% 
% Other 1% 1% 
% White 8% 20% 
Average GPA 22 3.0 
% GPA >= 27 4% 62% 
Average Attendance Rate 90% 94% 
% Attendance >= 94% 32% 62% 
% Proficient 10th Grade MCAS ELA 51% 80% 
% Proficient 10th Grade MCAS Math 54% 82% 
‘% with AP Course 29% 63% 
% with AP Exam 21% 61% 
% taking SAT 64% 86% 
% with MassCore 14% 40% 
% with MassCore + AP 1% 21% 
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